Skip to content

Move python dependencies inside a virtualenv #7403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

Move python dependencies inside a virtualenv #7403

wants to merge 1 commit into from

Conversation

damz
Copy link
Contributor

@damz damz commented Jan 22, 2017

No description provided.

@javiereguiluz
Copy link
Member

Error log:

Found 15695 commits.

Building application 'symfonydocs' (runtime type: php, tree: 83ed7d9)
  Generating runtime configuration.
  
  Installing build dependencies...
    Installing python build dependencies: virtualenv
  
  Moving the application to the output directory
  Executing build hook...
    New python executable in /app/.virtualenv/bin/python
    Installing setuptools, pip, wheel...done.
    make: Entering directory `/app/_build'
    sphinx-build -b html -c . -d ./doctrees   ../ ./html
    make: Leaving directory `/app/_build'
    Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
    Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
    make: sphinx-build: Command not found
    make: *** [html] Error 127
  
  E: Error building project: The build hook failed with status code 2. Aborted build.

E: Error building the project: Unable to build project, aborting deployment.

@damz
Copy link
Contributor Author

damz commented Jan 22, 2017

@javiereguiluz Thanks for that, I have access to it through the backend.

@damz
Copy link
Contributor Author

damz commented Jan 22, 2017

It looks like sphinx 1.3.6 is failing to build now :( Looking to see if upgrading it helps.

@damz damz mentioned this pull request Jan 22, 2017
Closed
@damz
Copy link
Contributor Author

damz commented Jan 22, 2017

This works. It looks like it started failing because we use pip --user, and that broke when, because of a change in dependencies, pip wanted to upgrade a system package.

So there are really two issues here:

  • The first is that the older version of pip that ships with Debian Jessie breaks in that case (it incorrectly tries to remove the system package);
  • The build dependencies that you have are not pinned, so it ended up braking because of changes of upstream dependencies (possibly when jinja2 ended up requiring markupsafe>=0.23, but that's a guess).

I also have a more intrusive version (pinning all the Python dependencies) at #7404.

@robfrawley
Copy link
Contributor

Looks like you came to the same conclusion of the platform.sh support team! ;-) I'm 👎 for the more intrusive version, though I do like offering a requirements file that allows for people to quickly get up and running, with just the sphinx-php requirement.

@javiereguiluz
Copy link
Member

Closing it in favor of #7404.

xabbuh added a commit that referenced this pull request Jan 27, 2017
@xabbuh
minor #7404 Pin Python dependencies and install them in a virtualenv …
8146f89 
…(damz)

This PR was submitted for the master branch but it was merged into the 2.7 branch instead (closes #7404).

Discussion
----------

Pin Python dependencies and install them in a virtualenv

(Alternative version from #7403)

Installing `sphinx` started to fail about a week ago, with `pip` incorrectly trying to remove a Python package shipped at the system level in Platform.sh environment.

It looks like it started failing because of a bug in the older version of `pip` that ships with Debian Jessie. We (Platform.sh) install global Python dependencies as user-site packages (using `pip --user`). We also ship system-site packages that we use for our own benefit.

So two issues here:

 * First, that older version of `pip`. We are going to look into upgrading it (a new batch of images based on Debian Stretch are cooking);
 * The build dependencies of the Symfony Docs are not pinned, which makes the build non-repeatable. I'm guessing that in this case it broke [when jinja2 ended up requiring markupsafe>=0.23][jinja2-commit] (but that's just a guess).

Fix the issue for good by installing in a `virtualenv` and pinning all the dependencies via `pip freeze`.

[jinja2-commit]: pallets/jinja@5453db1

Commits
-------

75e7177 Pin Python dependencies and install them in a virtualenv
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Status: Needs Review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@damz @javiereguiluz @robfrawley @carsonbot